DOCUMENT RESUME 



ED 429 472 



HE 031 847 



AUTHOR 

TITLE 



PUB DATE 
NOTE 
PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Wang, Aimin 

Developing a Measuring Instrument To Study Student 
Perceptions of Good College Teaching from a Cross-Cultural 
Perspective . 

1998-00-00 

12p. 

Reports - Research (143) -- Tests/Questionnaires (160) 

MFOl/PCOl Plus Postage. 

^College Instruction; Cross Cultural Studies; *Cultural 
Differences; Educational Attitudes; *Educational Quality; 
Higher Education; Questionnaires ; * Student Attitudes ; 
^Student Evaluation of Teacher Performance; *Test 
Construction; Test Reliability; Test Validity 



ABSTRACT 



This study sought to develop a valid and reliable instrument 
to measure student perceptions of good college teaching from a cross-cultural 
perspective. A total of 63 graduate and undergraduate students from various 
ethnic and racial groups (Caucasian, African American, Hispanic American, and 
Asian American) completed a set of open-ended Cfuestions regarding their 
perceptions of good teaching practices and their perceptions about learning. 
The results of these responses were used to develop a closed-end survey 
questionnaire that focused more narrowly on identifying the similarities and 
comparing the differences in perceptions of students from different cultural 
backgrounds. Test-retest reliability was examined using a group of 30 
students who completed the survey twice in a 2 . 5 week interval. A copy of the 
questionnaire is included. (MDM) 
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Abstract 

The purpose of this study is to develop a valid and reliable measuring instrument to study 
student perceptions of good college teaching from a cross-cultural perspective. Few studies have 
assessed perceptions of good college teaching from students of different cultural backgrounds. This 
is partially because of the lack of a measuring instrument to conduct this kind of research. The 
present study attempted to take steps to provide researchers with a useful tool to facilitate their 
studies of this aspect. 

Two phases of study were conducted to develop this instrument. A set of open-ended 
questions was used in the first phase and the results gathered from the first phase were used to 
develop the close-ended items in the second phase to make the instrument more focused. Content 
validity and test-retest reliability were examined. 
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The United States is changing at a rapid speed toward a multi-cultural society. According to 
the prediction of Corts (1994), the minority population will increase much faster than Caucasian. 
For example, while there will only be a 25% population increase in Caucasians from 1990 to 2030, 
there will be a 68% increase in African-American population and a 187% in Hispanic-American 
population. Understanding the needs of the students from different cultural background, therefore, 
becomes an increasingly important issue in college and university teachings. This in turn indicated 
the needs of studying student perceptions of good college teaching from a cross-cultural 
perspective. 

Many college instructors consider teaching a college class with students from a variety of 
cultural backgrounds both rewarding and challenging. According to Tomic (1996), teaching in a 
multi-cultural classroom gives instructors the opportunity to revise their teaching methods, 
reappraise their curriculum, develop professional skills, and take on a student-centered approach. 
However, it has been suspected that people from various cultures may not hold similar expectations 
regarding teacher and student roles (McCargar, 1993). If a teacher uses instructional methods that 
are unfamiliar to the students or does not help students adjust to a new culture, he or she potentially 
risks having withdrawn or unhappy students because of this violation of expectations (McCargar, 
1993). In multicultural classes, teachers could either become familiar with their students' 
expectations and use methods consistant with those expectations or help students change their 
expectations (McCargar, 1993). One way to do this is to develop a better understanding of how 
students from different cultures perceive effective teaching. 

In the past, few studies have assessed perceptions of good college teaching from students of 
different cultural backgrounds. This is partially because of the lack of a measuring instrument to 
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conduct this kind of research. The present study attempted to take steps to provide researchers a 
useful tool to facilitate their studies of this aspect. 

In 1992 Watkins and Akande indicated that although instruments measuring teaching 
effectiveness could be used to measure teaching effectiveness in other cultural settings, the distinct 
components underlying evaluations of teaching effectiveness at universities in the United States 
may not be the same in other cultures. Thus, while a given instrument may be reliable, it is doubtful 
whether there is cross-cultural validity to a model of underlying teaching effectiveness. Later in a 
study replicating the findings of Marsh (1993), Watkins (1994) investigated the validity of the 
models of teaching effectiveness developed in the United States by examining findings from six 
different countries of different economic, cultural, and philosophical traditions. In this study, 
students were asked to indicate up to five items that they considered most important for evaluating a 
given teacher and to indicate the items that were inappropriate for evaluating teaching. Watkins’ 
research provided evidence supporting the reliability and appropriateness of the instruments. 
Furthermore, there were similar patterns of item salience and differentiation between "good" and 
"poor" lecturers. These results thus indicated validity of the aspects underlying the model of 
teaching cross-culturally. 

Overall, teaching effectiveness is multifaceted and instruments focusing on a single overall 
score are likely to be inadequate (Watkins, 1994; Marsh, 1993; Watkins & Akande, 1992). Failure 
to separate components of what makes an effective instructor (e.g., organization, enthusiasm) can 
lead to conflicts in research findings and inadequate information for "diagnostic and decision- 
making purposes" (Watkins, 1994). 

A study by Nadal (1980) examined the beliefs about both college student and teacher roles. 
Results of this study found both agreement and disagreement among subjects on belief about the 
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roles of teachers and students. Likewise, Bail and Mina's (1981) study on Filipino and American 
students found that some aspects of teacher role were agreed upon, but others differed. For 
example, Filipino students rated characteristics such as authoritarianism and personal appearance as 
significantly more important for effective teaching than did American students. Furthermore, 
situations in which students are given more responsibility for assignments may be confusing and 
perhaps somewhat negatively perceived by the Filipino student (Bail & Mina, 1981). 

In assessing prior research on the topic, McCargar (1993) found American and Chinese 
students differed in role concepts they had about professors (Sun, 1 964). Another study found that 
British and American teachers held different beliefs about teacher roles than Chinese students and 
teachers (Bergaman,1984). 

In McCargar's study (1993), it was surprising to find that Japanese students indicated that 
teachers should encourage students to disagree with them, while Chinese and Indonesians disagreed 
with this idea. Other groups only mildly agreed with the statement. Such a diversity of student 
expectations could clearly become a barrier in classes where the open exchange of ideas is 
encouraged. 

Tomic (1996) describes the undergraduate years as a process of stress-adaptation-growth 
and states that for those who also find themselves in an unfamiliar cultural environment, this 
process can be particularly difficult. Additionally, the emotional and cognitive development of 
students is as diverse as the backgrounds from which they come (Tomic, 1996). Essentially, it is the 
professors' responsibility to understand their students' process of acculturation to the academic 
community. The researchers indicate that if one is knowledgeable about similar and conflicting 
value orientations of certain cultures, then cross-cultural differences in perceptions of effective 
teaching may become predictable. This, in turn, can have significant implications for instructors 
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concerned with students' academic adjustment. The purpose of this study is to develop a valid and 
reliable measuring instrument to study student perceptions of good college teaching from a cross- 
cultural perspective. 

Methods and Procedures of Developing This Instrument 

Two phases of studies were conducted to develop this instrument. During the first phase, 63 
college students, which included both graduate and undergraduate students of different cultures 
such as Caucasian, African-American, Hispanic American and Asian American, completed a set of 
open-ended questions regarding their perceptions of good college teaching practices and then- 
perceptions about learning. Based on the results of their responses, a close-ended survey was 
developed to make this instrument more focused on identifying the similarities and comparing the 
differences on the perceptions of students from different culture backgrounds. The content validity 
of this instrument was examined by students' responses and the instructors with cross-cultural 
teaching experiences and the responses were positive. Test-retest reliability was examined by a 
group of 30 students who took this survey twice in a two and a half week interval. The overall test- 
retest reliability coefficient was about 0.9. In addition, this revised instrument has been used in 
several studies to compare the perceptions of Caucasian students with those of African American 
students and Asian students and the findings from these studies were found to be meaningful. The 
findings also supported much of the intuitive knowledge of the people in this field and they have 
been reported in professional conferences. 

Description of the Measurement Instrument 

Five major aspects of teaching and learning were measured in five separate sections (See 
attached survey): 
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In the first section of the survey, the subject was asked basic demographic information 
including continent and country of origin, ethnicity, gender, amount of time spent living in the 
United States, year in college, major, number of graduate and undergraduate courses completed, 
and current course load. 

The second part of the survey contained questions on aspects of a good course, aspects of a 
disliked course, aspects of a good teacher, and aspects of a good student. In the initial version of the 
survey, open-ended questions were utilized. A content analysis was conducted on these responses 
and in the revised version of this survey, those most frequently stated responses were listed. 

Subjects were asked to read the statements and select the statements that were, in their opinion, the 
most important and least important statements. On the Likert scale to the right of each statement, 
the most important statement received a rating of "5" and the least important statement receives a 
rating of "1". The subject then evaluated the other statements using the most and least important 
statements as the standard. Any number from 1-5 could be selected for the remaining statements. 
The forced choice aspect of the survey encouraged a distribution of scores rather than subjects 
selection of all 5's or all I's for the statements. Thus, the sensitivity of the scale was increased. 

Subjects then rated from 1-7 the difficulty level experienced when completing the previous 
four questions. They also estimated the average number of hours spent studying each week for a 
three credit hour course and were asked to rate themselves on a scale from 1-7 on the kind of 
student they were (poor to excellent). Next, subjects indicated how much they would like an 
average class if the course work were doubled and how much they would like an average class if 
the course work were cut in half. 

In the next section of the survey, students rated, on a scale of 1 -7 (1 =least desirable, 7=most 
desirable) what they wanted to get from a course (to earn an "A", easy credit, to learn the material 
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well, to be able to use the knowledge); what they would do if they discovered a course was going to 
be extremely difficult (study extra hard, drop the course, blame yourself, blame the instructor); and 
the major reasons a course is considered to be easy, challenging, or just right (the professor, the 
content, the time needed for studying, the amount of knowledge learned). 

In the final section of the survey, subjects considered three major courses that they 
considered to be 1) easy, 2) challenging, and 3) just right. For each class, subjects specified the 
percentage of content they could understand when discussed in class, the percentage of the material 
they could learn, the percentage of the material they could understand before the exams, the 
percentage of the time they were frustrated in class, the percentage of courses they desired to have 
at that difficulty level in a given semester, and the percentage of courses at that level that they 
enjoyed. Subjects also indicated the number of hours needed to study for a class of that difficulty 
level each week, and the number of hours needed to study for an exam in that class. 

Educational Importance of the Work 

The educational importance of this piece of work is obvious. It is directly related to 
understanding students' perceptions of good teaching practice and therefore provides instructors 
knowledge of the expectations of their students. In addition to this practical importance, studies 
using this instrument will enable researchers to gather more knowledge about students' perspectives 
and expectations of teaching in general. The similarities and differences found in the research 
studies can then be used as guidelines for instructors to understand their students' needs better, 
quicker, and more accurately. 
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Pitas* send the completed form to: 

Aimin Vfang, EDP, Miami University, Oxford, OH 4S0S6 

Please circle a choice/choices in each statement or complete the statement 



♦ lam from (North America, South America, Africa, Asia, Europe, Australia, Other ]. 

♦ My home country is . 

♦ lam a(n) [Caucasian, Native American, African American, Hispanic American, Asian American, 

Hispanic^ Asian, Other ]. 

♦ I have been in the US for about years and months. This is about % of my life. 

♦ I am a [male, female]. 

♦ I am a [ 1st, 2nd, 3rd, 4th, ] year [undergraduate, master, Ph.D.] student 

♦ My major is . 

♦ I have completed about (# of ) undergraduate courses in the US 

and ! have completed about (it oO graduate courses in the US. 

♦ lam taking courses this semester/quarter. 



Please follow the following steps to respond to the questions below. 

1. Read all the statements under each question and select the statement that is, in your perception, the most 
important to the question. Circle 5 for that statement. 

2. Select the least important statement. If all of the statements are very important to you, compare them 
and find the least important one among them. Circle 1. 

3. Respond to the other statements using the above choices as your standards. You may choose any 
number for the remaining statements, including 1 and S. 



Aspects of a good course ; 

• I learn a lot 1 

• It is interesting and enjoyable 1 

• It is applicable to my career and real-life situations 1 

• It has a good instructor 1 

• It has clarity 1 

• Students are involved 1 

• It has a good means of evaluation 1 

• It is challenging 1 



2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 
2 3 4 5 



Aspects of a disliked course ; 

I do not learn much 

It is boring 

all lecture 

It is useless 

It has a poor instructor 

It lacks clarity 

There is not interaction among the students 
or between the students and the instructor 
It has unfair means of evaluation 
It has too much work 




Aspects of a good teacher: H^She 

...is knowledgeable 
...is enthusiastic 
...has good teaching ability 
...is available 

...has positive relations with students 
...is flexible 

...Other 

(over) (over) (over) 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 3 

1 2 3 4 5 

1 2 3 4 3 

1 2 3 4 5 

(over) (over) (ovss") 
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Aspects of a good student : 

• Involved in learning and learning activities 

• Motivated to learn 

• Demonstrates effort and enthusiasm in learning 

• Learns the material 



12 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 



♦ On a scale from 1-7(1 = easy, 7 = difficult), rate the difficulty level you experienced when completing 
the five questions immediately above this one: 

about a good course 

about a good teacher 

about a good student 

about a disliked class 



♦ On the average* how many hours do you expect to study each week, in addition to class time, for a 

three credit hours course? Hours 

♦ On a scale from 1-7 (1= poor , 7 = excellent), assume your evaluation for a course is 4. If the out-of 

class study time is doubled , how do your rate the course now? 

♦ How do you rate this course if the out-of-class study time is cut in half ? 

♦ On a scale from 1-7 (l=a poor student, 7 = an excellent student), where do you place yourself? 



Please rate each of the following statements from 1 to 7 (1 = least desirable, 7 = most desirable)* You may use the 
same rating more than once* 

♦ Which of the following do you want to get the most from a course? 

To earn an A. Rate 

Easy credit 

To learn the material well 

To be able to use the knowledge 

Other (specify) 

♦ If you discover a course you just began is going to be extremely difficult, would you rather: 

Study extra hard Rate 

Drop the course 

Blame the instructor 

Blame yourself 

Other (specify) 

♦ The major reasons that you consider a course to be easy, challenging, or just right are: 

The professor Rate 

The content 

The time needed for studying 

The amount of knowledge need to be learned 

Other (Specify) 

Think about three courses within your m^or that you consider to be 1) easy, 2) challenging, and 3) just 
right for you. Rate each of them on the following aspects by filling in the blank with an appropriate 
percentage (question #1 to #7) or number of hours (question #8 & #9). 

Example: Easy Course Challenging Just Right 

I attend class about 30% 50% 20% of the time in a semester 



I spend 


2 


10 


5 


hours reading textbook in this class 


Aspects 


Easy Course 


Challenging 


Just Right 




1* I can understand 








% of the content when they are discussed in class 


2* I can learn 








% of the materials I need to learn 


3* I can understand 








% of the material before the exams 


4. About 








% of the time I am frustrated in the cl^ 


5* I prefer to have 








% of my courses of that difBculty level in a given 
semester 


6* About 








% of the time I look forward to attending 


7. 1 enjoy 








% of a course like that 


8. 1 need to spend 








hours studying after the class each week 


9. 1 need to spend 








hours studying for an exam in this class 



Please send the completed form to: 

Aimin Wan^ EDP, Miami University, Oxford, OH 45056 
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